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1.  Introduction 


The  Battlescale  Forecast  Model  (BFM)  is  an  operational  mesoscale  forecast  model  developed  at 
the  U.S.  Army  Research  Laboratory.1  It  has  been  extensively  used  to  make  short-range  forecasts 
of  atmospheric  conditions  as  a  component  in  both  the  Integrated  Meteorological  System 
(IMETS)  and  the  Computer  Assisted  Artillery  Meteorology  System.  For  prognostic  calculation, 
the  BFM  uses  the  Higher  Order  Turbulence  Model  for  Atmospheric  Circulation  developed  by 
Yamada.2 

After  Operation  Enduring  Freedom  started,  a  validation  study  of  the  BFM  over  model  domains 
in  western  Asia  became  necessary  to  prepare  for  use  of  the  BFM  over  Afghanistan.  It  would 
have  been  ideal  to  run  the  BFM  on  the  model  domain  over  Afghanistan;  unfortunately,  no 
meteorological  data  are  presently  available  over  Afghanistan.  Therefore,  two  model  domains  in 
the  same  region,  one  over  Iran  and  the  other  over  Pakistan,  were  selected  for  the  BFM  validation 
study.  Figure  1  shows  approximate  locations  of  model  domains  used  for  this  study. 


Figure  1.  Approximate  locations  of  model  domains  used  for  the  BFM  validation  study.  Each 
square  covers  the  500-  x  500-km  area. 


1  Henmi,  T.,  and  R.  Dumais,  Jr.  “Description  of  the  Battlescale  Forecast  Model  (BFM).”  ARL-TR-1032,  U.S.  Army 
Research  Laboratory,  Aberdeen  Proving  Ground,  MD,  1 998. 

2  Yamada,  T.,  and  S.  Bunker.  “A  Numerical  Model  Study  of  Nocturnal  Drainage  Flows  With  Strong  Wind  and  Temperature 
Gradients.”  Journal  of  Applied  Meteorology,  vol.  28,  pp.  545-554, 1999. 
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For  this  study,  scientists  of  the  Physical  Sciences  Laboratory  of  New  Mexico  State  University 
ran  the  BFM  twice  a  day,  at  00  and  12  universal  time  coordinate  (UTC),  and  archived  observed 
data  available  over  the  model  domains.  The  BFM  output  and  surface  observation  data  were 
compared  to  validate  the  BFM.  The  study  was  based  on  the  data  obtained  during  October  and 
November  2001 .  (The  validated  BFM  was  the  one  on  the  U.S  Army  EMETS  ABCS  6  2  1  0.) 
Meteorological  data,  including  surface  and  upper  air  data  and  Navy  Operational  Global 
Atmospheric  Prediction  System  (NOGAPS)  forecast  data,  were  obtained  from  the  U.S.  Air  Force 
Weather  Agency  using  the  Air  Force  Tactical  VSAT  System. 

During  the  same  period,  surface  wind  field  analysis  data,  model  output  by  NOGAPS,  Aviation 
Model  (AVN),  and  Mesoscale  Model  Version  5  (MM5),  were  obtained  from  the  U.S.  Navy 
Meteorology  and  Oceanography  Command  through  the  Internet  using  the  METCAT  System. 
These  data  were  compared  with  surface  observation  data  archived  as  previously  described. 

The  purpose  of  this  report  is  to  describe  the  results  of  statistical  comparisons  of  the  BFM  forecast 
data  with  observation  data  and  those  of  the  NOGAPS,  AVN,  and  MM5,  with  observation  data 
over  western  Asia.  It  is  noted  that  the  present  study  is  limited  to  the  surface  meteorological  data 
and  that  because  of  the  limited  number  of  observation  sites  and  data,  the  results  should  be 
regarded  as  qualitative. 


2.  Review 


In  the  following,  the  validation  studies  done  in  the  past  are  briefly  reviewed.  The  forecast  skill 
of  the  BFM  was  compared  with  that  of  MM5  by  applying  the  models  to  the  domain  of  White 
Sands  Missile  Range,  NM,  which  covers  a  167-  x  167-km  area  (51  x  51  grid  points,  with 
3.33-km  grid  spacing).  Meteorological  parameters  forecasted  by  the  models  were  compared  with 
observed  data.  The  comparison  study  showed  that  the  forecast  skills  of  the  BFM  were 
comparable  to  those  of  the  MM5.  Surface  temperature  forecasted  by  both  the  BFM  and  MM5 
agreed  well  with  observed  values.  Both  the  BFM  and  the  MM5  showed  difficulties  for 
forecasting  the  relative  humidity.  For  wind  parameters,  both  models  tended  to  predict  wind 
speed  less  than  that  observed,  but  BFM  calculations  produced  lower  wind  speed  than  the  MM5. 
For  wind  direction,  the  BFM  resulted  in  better  forecasts  than  the  MM5.3 

The  BFM  in  operational  mode  on  the  IMETS  has  been  extensively  used  over  the  500-  x  500-km 
model  domain,  with  grid  spacing  of  10  km.  A  statistical  evaluation  of  the  BFM  in  operational 
mode  was  conducted  for  cases  during  a  30-day  period,  with  over  three  different  model  domains 


3 

Herani,  T.  Comparison  and  Evaluation  of  Operational  Mesoscale  MM5  and  BFM  Over  WSMR ARL-TR-1476  U  S 
Army  Research  Laboratory,  Aberdeen  Proving  Ground,  MD,  2000. 
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having  different  terrain  complexities  and  climate  (Colorado,  Washington,  and  Florida).4  The 
model  calculations  were  initialized  with  the  following  three  different  sets  of  initial  conditions: 

(1)  NOGAPS  +  upper  air  data  +  surface  data, 

(2)  NOGAPS  +  upper  air  data,  and 

(3)  NOGAPS. 

Forecast  data  for  24-hr  periods  were  statistically  compared  with  surface  observation  data  by 
calculating  parameters  such  as  mean  difference  (MD),  absolute  difference  (AD),  root  mean 
square  error  (RMSE),  root  mean  square  vector  error  (RMSVE),  and  correlation  coefficient  (CC). 

For  all  three  model  domains,  the  temperature  fields  of  BFM  initialized  with  (1)  and  (2)  were 
statistically  better  than  those  initialized  with  (3).  For  Colorado  and  Washington  model  domains, 
the  BFM  showed  clear  tendencies  of  forecasting  dew  point  temperature  lower  than  those 
observed  throughout  the  24-hr  forecast  period.  However,  for  the  Florida  model  domain, 
forecasts  of  dew  point  temperature  were  higher  than  observed. 

Three  different  types  of  initialization  data  did  not  produce  significantly  different  wind  fields 
throughout  the  24-hr  forecast  period.  The  value  of  MD  for  wind  speed  was  in  the  range  of  0  to 
1  m/s.  And  those  of  AD  were  also  between  0  and  1  m/s  for  three  model  domains  throughout  the 
24-hr  forecast  period. 

For  Colorado  and  Washington  model  domains  where  terrain  was  more  complex  than  Florida,  the 
use  of  the  BFM  improved  temperature  forecasts  over  those  of  NOGAPS  and  Navy  Operational 
Regional  Atmospheric  Prediction  System  (NORAPS).  For  the  Florida  domain,  no  significant 
improvements  in  temperature  forecasts  were  found.  Similarly,  the  BFM  produced  better  wind 
fields  than  NOGAPS  and  NORAPS  over  Colorado  and  Washington. 


3.  Model  Domains  for  Present  Study 


The  following  two  BFM  domains  were  chosen  for  the  presort  study: 

(1)  Iran,  centered  at  34.50  N  and  52.98  E  and 

(2)  Pakistan  centered  at  35.58  N  and  72.72  E. 


Each  model  domain  covers  a  500-  x  500-km  area,  with  10-km  horizontal  grid  spacing  and 
51x51  horizontal  grid  number.  The  vertical  depth  of  the  model  is  7  km  above  the  highest  point 
of  the  model  domain,  with  16  vertical  layers. 


4  Henmi,  T.  “Evaluation  Study  of  an  Operational  Mesoscale  Forecast  Model  Over  Three  Climatologically  Different  Areas.” 
ARL-TR-1034,  U.S.  Army  Research  Laboratory,  Aberdeen  Proving  Ground,  MD,  2000. 
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The  complexities  of  the  two  model  domains  are  summarized  in  Table  1.  For  comparison,  the 
complexity  data  of  the  same  size  area  centered  at  Kabul,  Afghanistan  is  also  given. 


Table  1.  Terrain  complexities  of  the  three  domains. 


Terrain 

(m) 

Iran 

Pakistan 

Afghanistan 

Mean  elevation 

1237 

2991 

2232 

Max.  height 

3744 

6717 

6413 

Min.  height 

-28 

247 

173 

Standard  dev. 

642 

1513 

1153 

As  seen  in  Table  1,  the  Pakistan  model  domain  is  the  most  complex,  containing  high  peaks  of  the 
Hindu  Kush  Mountain  range  in  the  northern  part  of  the  model  domain.  Compared  to  the 
Pakistan  model  domain,  the  Iran  model  domain  is  more  flat,  with  the  Caspian  Sea  below  zero 
level  at  the  northern  part  of  the  domain.  As  far  as  the  complexity  is  concerned,  the  500-  x  500- 

km  area  of  Afghanistan  centered  at  Kabul  is  fairly  complex,  similar  to  the  Pakistan  model 
domain. 

Figures  2  and  3  are  the  elevation  contour  maps  of,  respectively,  the  Iran  and  Pakistan  model 
domains.  In  these  figures,  the  outside  squares  cover  the  NOGAPS  analysis  and  data  composite 
area  of  800  x  800  km,  and  the  inside  square  covers  the  model  domains  of 500  x  500  km.  In 
these  figures,  N  and  S  represent  the  locations  of  the  input  data  for  model  initialization  of  the 
model  calculation.  There  were  a  few  upper  air-sounding  stations  in  the  surrounding  areas  for 
both  domains,  but  because  of  incomplete  form  of  the  data  sets,  they  were  not  used  to  initialize 
the  BFM  throughout  the  study  period  for  both  domains. 

In  Figure  2,  there  are  seven  stations  of  surface  observations  which  are  used  to  compare  with  the 
BFM  forecast  data.  However,  there  is  only  one  surface  station  in  Figure  3  used  for  this  study. 
Therefore,  the  statistical  results  obtained  from  the  Pakistan  model  domain  should  be  regarded  as 
semiquantitative. 


4.  Method 


4.1  Study  Period 

The  study  was  initiated  immediately  after  the  commencement  of  Operation  Enduring  Freedom. 
Table  2  shows  the  date  and  initialization  times  of  the  BFM  for  this  study.  For  most  of  the  days 
when  the  BFM  was  run,  the  BFM  was  initialized  at  00  and  12  UTC  for  24-hr  forecasts. 
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Notes:  The  outside  area  covers  800  x  800  km,  and  the  inside  area  covers  500  x  500  km. 

N  =  NOGAPS  data  points. 

S  =  surface  observation  locations. 

Figure  2.  Elevation  contours  of  the  Iran  model  domain. 

4.2  Forecast  Method 

Forecast  calculation  of  the  BFM  is  currently  done  as  follows: 

•  From  NOGAPS  forecast  data,  horizontal  wind  vector  components,  temperature,  dew  point 
temperature,  and  geopotential  height  at  13  different  pressure  levels  (1000,  975, 925, 900, 
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Notes:  N  -  NOGAPS  data  points. 

S  =  surface  observation  locations. 


Figure  3.  Elevation  contours  of  the  Pakistan  model  domain. 


850, 700,  500, 400,  300, 250, 200, 150,  and  100  mb)  are  obtained  for  the  forecast  periods 
of  0, 6, 12, 18,  and  24  hr.  These  data  are  interpolated  to  the  81  x  81  grid,  with  grid  spacing 
of  10  km  at  each  pressure  level  for  the  forecast  periods.  The  data  is  then  vertically 
interpolated  from  the  pressure  levels  to  the  BFM’s  height  levels  to  produce  four- 
dimensional  fields  (x,  y,  z,  t)  of  the  input  data  for  initialization  and  lateral  boundary  data. 


Table  2.  Date  and  initialization  time  of  the  BFM  run. 


Date 

Iran 

Pakistan 

00  UTC 

12  UTC 

00  UTC 

12  UTC 

October  2001 

5,9, 10,11, 
15, 16, 17, 18, 
23,24 

12, 15,  16,  17, 
18,  23, 24 

9, 10, 11, 15, 

16, 17,  18, 23, 
24 

9,10,11,  15, 

16, 17, 18, 22, 
23,24 

November  2001 

14, 15, 19, 20, 
26,27,28, 29, 
30 

8, 15, 18, 27, 
28, 29, 30 

14, 18, 19, 20, 
26, 27, 29, 30 

8, 13, 15, 19, 

20, 27, 28, 29, 

30 

•  Suppose  a  forecast  calculation  is  initialized  at  time  to.  A  precalculation  will  start  at  time 
to  -  3  hr;  and  for  3  hr  from  to  -  3  to  to,  the  model  fields  are  dynamically  adjusted  to  the 
initial  fields  by  the  nudging  method. 

•  The  hourly  lateral  boundary  condition  data  between  two  different  forecast  periods  are 
calculated  by  a  linear  interpolation  method. 

•  From  time  t  to  t  +  1,  the  data  for  t  +  1  are  assimilated  in  for  1  hr;  this  process  is  repeated  for 
an  entire  forecast  period. 

4.3  Data  Comparison  Method 

After  the  forecast  calculation  is  completed,  the  following  bilinear  interpolation  is  conducted  to 
obtain  the  BFM  data  at  the  surface  observation  locations: 

•  Suppose  a  surface  observation  location  (x’  and  y’)  is  surrounded  by  four  BFM  grid  points. 
An  interpolated  value  q>’  of  an  arbitrary  variable  cp  at  (x’,  y’)  is  calculated  using  a  bilinear 
interpolation  method  as  follows: 

<Pi  =  <P(i>  j)  +  (x*  -  x)[<p(i  +  1,  j)  -  cp(i,  j)].  (1) 

<P2  =  <p(i,  j  +  1)  +  (x*  -  x)[cp(i  +  1,  j  +  1)  -  <p(i,  j  +  1)].  (2) 

<p’(x\  y’)  =  <pi  +  (y’-  y)|>2  -  q>i3-  (3) 


Here  (i,  j)  is  the  southwest  grid  point  of  the  four  grid  points  surrounding  a  surface  observation 
location  (x\  y’),  and  (x,  y)  is  the  location  for  the  grid  point  (i,  j);  <p(i,  j)  is  an  arbitrary  variable  at 
(x,y). 


5.  Statistical  Parameters 


The  following  statistical  parameters  between  the  BFM  forecast  data  and  surface  observation  data 
are  calculated  using  the  data  available  in  the  cases  given  in  Table  2.  The  parameters  are 
calculated  for  the  forecast  periods  0, 3, 6, 9, 12, 15, 18, 21,  and  24-hr  and  for  the  entire  period. 
Statistical  parameters  are  calculated  for  temperature,  relative  humidity,  wind  speed,  and 
horizontal  wind  vector  components,  u  and  v. 


5.1  Mean  Difference 


MD  = 


mn 

Here  the  subscripts  o  and  p  represent  observation  and  prediction,  respectively.  The  subscript  i 
represents  the  i*  surface  station,  and  the  subscript  j  the  j*  forecast  day.  n  is  the  number  of 
surface  stations,  and  m  the  total  number  of  forecast  days.  A  nonzero  MD  indicates  bias.  For 
instance,  if  the  MD  value  is  positive,  it  indicates  that  the  model  tends  to  over-forecast. 


(4) 


5.2  Mean  Absolute  Difference 


mn 


(5) 


5.3 


Root  Mean  Square  Error  (RMSE) 


RMSE  = 


(6) 


Good  agreements  between  observation  and  forecast  are,  in  general,  related  to  smaller  values  of 
AD  and  RMSE. 


5.4  Root  Mean  Square  Vector  Error  (RMSVE) 


RMSVE 


V  mn 


This  parameter  measures  the  differences  of  both  wind  speed  and  direction.  Again,  good 
agreements  of  wind  vectors  are  related  to  small  values  of  the  RMSVE. 


(7) 
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5.5  Correlation  Coefficient 


CC  = 


m  n 


i='M 


- 

Vi 


PJJ 

/=!  jm  1  1-1  >=1 


(8) 


Here 


ya, 

i,j  ^ o,i,j  *o  ’ 

(9) 

and 

yp*u 

i=Xp,U-XP  ’ 

(10) 

and  x0  and  xp  are  the  means  of  observed  and  forecast  values,  respectively. 


6.  BFM  Validation  Results 


6.1  Statistical  Parameters  at  Different  Forecast  Periods 

Tables  3-10  show  computed  statistical  parameters  of  surface  meteorological  parameters  using 
BFM  calculated  and  observed  values.  No  statistical  parameters  are  calculated  for  wind 
parameters  for  the  Pakistan  model  domain  due  to  the  lack  of  surface  observations. 

6.2  Scatter  Diagrams  of  Model  Forecast  Against  Observations 

The  following  scatter  diagrams  (Figures  4-10)  are  obtained  by  plotting  all  the  data  for  all 
forecast  periods  and  days.  As  can  be  seen,  there  is  no  comparison  made  for  forecast  periods 
beyond  18  hr,  and  no  comparison  was  made  for  wind  components  for  the  Pakistan  model 
domain. 

As  noted  before,  it  is  unfortunate  that  there  are  not  enough  observed  data  of  wind  speed  and 
direction  over  the  Pakistan  model  domain  to  make  statistical  comparison  with  BFM  forecast 
data.  Even  for  the  Iranian  model  domain,  comparisons  could  not  be  made  for  a  forecast  period 
beyond  18  hr.  As  can  be  seen  from  Tables  3  and  4  and  Figures  4  and  5,  there  are  good 
agreements  for  temperature  between  the  BFM  forecasts  and  surface  observation  for  both  model 
domains,  with  the  CC  values  greater  than  0.8.  The  MD  values  in  Tables  3  and  4  show  that  the 
BFM  slightly  underforecasted  temperature  values. 

For  relative  humidity  forecasts,  the  CC  value  of  0.65  for  the  entire  data  set  over  the  Iran  model 
domain  should  be  regarded  as  good,  considering  the  lack  of  input  data  for  model  initialization 
over  this  region.  Note  that  for  this  study,  there  were  no  upper  air-sounding  data  used.  In  a 
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Table  3.  Statistical  parameters  for  temperature  (C)  over  the  Iran  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

MD 

AD 

RMSE 

CC 

0 

53 

-.98 

3.24 

4.92 

.81 

3 

95 

-1.44 

3.30 

4.92 

.81 

6 

91 

-.96 

3.14 

3.86 

.81 

9 

27 

-2.02 

3.65 

4.70 

.71 

12 

47 

-2.46 

2.68 

4.70 

.75 

15 

32 

-4.25 

4.39 

5.34 

.72 

18 

5 

21 

0 

24 

1 

Entire  period 

351 

-1.65 

3.27 

4.64 

.81 

Table  4.  Statistical  parameters  for  temperature  (C)  over  the  Pakistan  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

MD 

AD 

RMSE 

CC 

0 

10 

-1.79 

1.86 

2.90 

.94 

3 

37 

3.44 

3.95 

5.15 

.81 

6 

36 

2.46 

2.94 

3.42 

.84 

9 

14 

.51 

3.05 

3.61 

.89 

12 

4 

15 

11 

1.29 

1.79 

2.26 

.85 

18 

7 

21 

1 

24 

0 

Entire  period 

120 

-1.79 

1.86 

2.90 

.94 

similar  validation  study  of  the  BFM  over  model  domains  in  the  United  States,  the  CC  for  dew 
point  temperature  between  forecast  and  observation  were  0.78  for  the  Colorado  area,  0.44  for 
Washington,  and  0.26  for  Florida.3 

For  wind  parameters  (speed  and  u  and  v),  the  values  of  the  MD  are  slightly  negative,  implying 
that  the  BFM  tends  to  underforecast  the  magnitudes  of  wind  parameters  (see  Tables  7  and  8  and 
Figures  8-1 0).  In  the  study  over  model  domains  in  the  United  States,  the  values  of  RMSVE 
were  about  3  m/s  or  smaller  throughout  the  24-hr  forecast  period.4  In  this  study,  as  can  be  seen 
in  Table  10,  the  RMSVE  is  about  4  m/s.  The  CC  values  of  this  study  are  also  slightly  smaller 
than  those  over  the  model  domains  in  the  United  States. 
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Table  5.  Statistical  parameters  for  relative  humidity  over  the  Iran  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

MD 

AD 

RMSE 

CC 

0 

53 

1.5 

6.4 

10.6 

.83 

3 

95 

-3.2 

10.4 

14.8 

.67 

6 

91 

-4.8 

10.5 

12.8 

.68 

9 

27 

-1.5 

10.1 

11.7 

.49 

12 

47 

.9 

10.8 

13.8 

.23 

15 

32 

3.2 

11.3 

15.4 

.22 

18 

5 

21 

0 

24 

1 

Entire  period 

351 

-1.8 

9.9 

13.4 

.65 

Table  6.  Statistical  parameters  for  relative  humidity  over  the  Pakistan  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

MD 

AD 

RMSE 

CC 

0 

10 

2.4 

3.1 

3.8 

.99 

3 

37 

-31.5 

31.5 

34.5 

.25 

6 

36 

-24.3 

25.1 

27.9 

.30 

9 

14 

-21.9 

22.5 

28.2 

.60 

12 

4 

15 

11 

-27.0 

27.0 

28.0 

.33 

18 

7 

21 

1 

24 

0 

Entire  period 

120 

-24.0 

25.1 

29.1 

.36 

Compared  to  North  America,  the  number  of  meteorological  observations  in  the  Western  Asia  is 
spatially  and  temporally  limited.  Therefore,  the  quality  of  NOGAPS  data,  which  is  used  for 
initialization  and  boundary  data  of  the  BFM,  may  be  more  important  over  western  Asia  than  over 
North  America.  The  present  validation  results  may  be  partially  attributed  to  the  NOGAPS  data 
over  western  Asia. 
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Table  7.  Statistical  parameters  of  wind  speed  (meters/second)  over  the  Iran  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

MD 

AD 

RMSE 

CC 

0 

39 

-2.0 

2.6 

3.27 

.37 

3 

53 

-.7 

1.8 

2.4 

.19 

6 

49 

-.9 

1.8 

2.3 

.32 

9 

15 

-.4 

2.3 

3.0 

.05 

12 

40 

-.8 

2.4 

3.0 

.14 

15 

19 

-.8 

1.3 

1.6 

.06 

18 

2 

21 

0 

24 

0 

Entire  period 

217 

-.5 

2.4 

3.0 

.35 

Table  8.  Statistical  parameters  of  wind  component  u  (meters/second)  over  the  Iran  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

MD 

AD 

RMSE 

CC 

0 

39 

-.3 

2.6 

3.1 

.60 

3 

53 

.4 

2.4 

2.9 

.35 

6 

49 

.2 

2.4 

2.7 

.35 

9 

15 

-1.0 

2.5 

3.0 

.46 

12 

40 

-1.4 

2.9 

3.6 

.51 

15 

19 

.6 

1.5 

1.9 

.37 

18 

2 

21 

0 

24 

0 

Entire  period 

217 

-.1 

2.5 

3.0 

.45 

7.  Comparisons  of  Surface  Wind  Analysis  Data  of  the  MM5,  AVN,  and 
NOGAPS 


Surface  wind  analysis  data  at  00  and  12  UTC  by  the  MM5,  AVN  and  NOGAPS  obtained  from 
the  Navy  through  the  internet  contain  the  gndded  data  covering  the  following  areas  and  days: 
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Table  9.  Statistical  parameters  of  wind  component  v  (meters/second)  over  the  Iran  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

MD 

AD 

RMSE 

CC 

0 

39 

-.3 

2.4 

3.0 

.47 

3 

53 

-1.3 

2.2 

2.7 

.40 

6 

49 

-.9 

2.4 

3.0 

.35 

9 

15 

.6 

2.8 

3.3 

.20 

12 

40 

.3 

3.0 

3.7 

.35 

15 

19 

-.2 

1.3 

2.5 

.1 

18 

2 

21 

0 

24 

0 

Entire  period 

217 

-.5 

2.4 

3.0 

.35 

Table  10.  RMSVE  (meters/second)  wind  vector  over  the 
Iran  model  domain. 


Forecast 

Period 

(hr) 

Number  of 
Observations 

RMSVE 

0 

39 

4.3 

3 

53 

3.9 

6 

49 

4.2 

9 

19 

4.0 

12 

40 

4.9 

15 

19 

3.3 

18 

2 

21 

0 

24 

0 

Entire  period 

217 

4.3 

•  MM5:  24°  E  -  75°  E,  and  16.4°  N  -  58.4°  N  with  0.403°  grid  spacing,  and 

•  9-day  data  from  1 5  October  to  25  October  2001 . 

•  AVN  and  NOGAPS:  53°  E  -  80°  E,  and  21°  N  -  45°  N,  with  1°  grid  spacing,  and 

•  1 7-day  data  from  1 5  October  to  30  November. 

Tables  1 1-13  are  statistical  parameters  calculated  for  surface  wind  analysis  data  against  observed 
data  for  the  MM5,  AVN,  and  NOGAPS,  respectively.  The  values  of  RMSVE  are  given  in  the 
description  of  each  table. 
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Figure  4.  Scatter  diagram  of  BFM  vs.  surface  observation  for 
temperature  over  the  Iran  model  domain. 


Figure  5.  Scatter  diagram  of  BFM  vs.  surface  observation  for 
temperature  over  the  Pakistan  model  domain. 


14 


Figure  7.  Scatter  diagram  of  BFM  vs.  surface  observation  for 
relative  humidity  over  the  Pakistan  model  domain. 
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Figure  8.  Scatter  diagram  of  BFM  vs.  surface  observation  for 
wind  speed  (meters/second)  over  the  Iran  model 
domain. 


Figure  9.  Scatter  diagram  of  BFM  vs.  surface  observation  for 
wind  vector  component  u  (meters/second)  over  the 
Iran  model  domain. 
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Figure  10.  Scatter  diagrams  of  BFM  vs.  surface  observation  for 
wind  vector  component  v  (meters/second)  over  the 
Iran  model  domain. 


Table  1 1.  Statistical  parameters  of  wind  speed  and  u  and  v  for  surface  wind  analysis  data  by  the  MM5; 
RMSVE  =  4.0  m/s. 


No.  of  data 

MD 

AD 

RMSE 

CC 

Speed  (m/s) 

473 

-.5 

1.7 

2.5 

.42 

u 

473 

.2 

1.7 

2.6 

.62 

V 

473 

.4 

2.0 

3.1 

.63 

Table  12.  Same  as  Table  1 1,  except  for  AVN;  RMSVE  =  3.3  m/s. 


No.  of  data 

MD 

AD 

RMSE 

CC 

Speed  (m/s) 

253 

.8 

1.9 

2.6 

.25 

u 

253 

.4 

1.4 

2.2 

.58 

V 

253 

.4 

1.7 

2.5 

.55 
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Table  13.  Same  as  Table  1 1,  except  for  NOGAPS;  RMSVE  =  4.0  m/s. 


No.  of  data 

MD 

AD 

RMSE 

CC 

Speed  (m/s) 

233 

-.9 

1.6 

2.1 

l_  .45 

u 

233 

1.3 

1.8 

2.7 

.47 

V 

233 

1.1 

1.9 

2.9 

.42 

Scatter  diagrams  of  model  analysis  data  of  wind  speed  and  u  and  v  are  shown  in,  respectively. 
Figures  1 1-13  for  the  MM5,  Figures  14-16  for  AVN,  and  Figures  17-19  for  NOGAPS. 

Fairly  good  statistical  values  previously  shown  should  have  been  expected  because  surface 

observation  data  used  for  comparison  might  have  been  used  to  derive  surface  analysis  data  by 
these  models. 

It  is  cautioned  that  the  statistical  results  obtained  for  these  models,  the  MM5,  AVN,  and 
NOGAPS,  shown  in  this  section  should  not  be  used  to  compare  quantitatively  to  those  of  BFM 
shown  in  the  previous  section,  or  those  of  other  models,  because  of  the  following  reasons: 

•  Model  domain  size  and  grid  are  different  from  each  other, 

•  Numbers  of  surface  data  used  for  comparison  are  different,  and 

•  Numbers  of  forecast  days  are  different. 


Figure  11.  Scatter  diagram  of  surface  wind  speed  analysis  data 
by  the  MM5  against  observation. 


18 


Analysis ( MM5 ) 
Figure  12.  Same  as  Figure  1 1,  except  for  u. 


Figure  13.  Same  as  Figure  11,  except  for  v. 


19 


Figure  14.  Scatter  diagram  of  surface  wind  speed  analysis  data 
by  AVN  against  observation. 
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8.  Summary  and  Consideration 


After  the  start  of  the  U.S.  military  operation  over  Afghanistan,  a  validation  study  of  the  BFM  in 
operational  mode  over  the  western  Asian  region  became  urgently  necessary. 

It  would  have  been  ideal  to  run  the  BFM  over  Afghanistan,  but  lack  of  the  availability  of 
meteorological  observation  data  forced  the  selection  of  model  domains  adjacent  to  Afghanistan. 
The  current  version  of  BFM  on  the  U.S.  Army  IMETS  ABCS_6_2_1_0  was  nm  over  Iran  and 
Pakistan  model  domains  and  evaluated  by  comparing  surface  forecast  data  to  surface  observation 
data.  Unfortunately,  the  availability  of  observation  data  was  far  less  than  ideal.  There  were  very 
few  upper  air-sounding  data  which  could  be  used  to  initialize  the  BFM  and  to  compare  to 
forecast  data.  For  surface  data,  there  were  seven  stations  in  the  Iran  model  domain  and  only  one 
station  in  the  Pakistan  model  domain.  Thus,  it  is  emphasized  that  the  results  obtained  in  this 
study  should  be  regarded  at  best  as  qualitative. 

•  For  surface  temperature,  the  BFM  forecast  data  produced  good  agreement  with  observed 
data  with  the  CC  of  0.81  for  the  entire  data  set  for  the  Iran  model  domain. 

•  Relative  humidity  forecasts  were  also  fairly  well  done  by  the  BFM,  with  the  CC  of  0.65  for 
the  Iranian  model  domain. 

•  The  BFM  showed  the  tendency  of  underforecasting  wind  speed  throughout  the  entire 
forecast  period,  with  the  MD  of  -0.5  m/s  for  the  Iran  domain.  Similarly,  forecasted  values 
of  horizontal  wind  vector  components  u  and  v  tended  to  be  smaller  than  observed  values. 
The  CC  for  wind  speed  and  u  and  v  were,  respectively,  0.35, 0.45,  and  0.35  for  data 
covering  the  entire  study  period  over  the  Iran  model  domain. 

Questions  as  to  whether  the  statistical  results  over  Iran  can  be  applied  to  Afghanistan  are  difficult 
to  answer.  As  shown  in  Table  1,  terrain  data  over  Afghanistan  showed  more  complexities  than 
the  one  over  Iran,  but  not  as  complex  as  the  one  over  Pakistan.  Over  the  Pakistan  model  domain, 
temperature  forecast  data  by  the  BFM  showed  fairly  good  agreement  with  observation,  but  based 
on  only  one  station  data.  In  a  previous  validation  study  of  the  BFM  over  different  climatological 
and  topographical  areas  in  the  United  States  (Colorado,  Washington,  and  Florida),  no  significant 
differences  in  statistical  parameters  for  surface  wind  parameters  were  obtained.4 

Western  Asia  was,  in  general,  not  a  good  region  to  conduct  mesoscale  model  validation  studies 
because  of  the  limited  amount  of  observed  meteorological  data.  Ideally,  the  validation  study 
should  have  been  done  over  the  areas  with  complex  topographical  feature  where  a  large  number 
of  surface  and  upper  air  data  were  regularly  available.  For  instance,  the  Utah  area  in  the  United 
States  may  be  a  proper  region  to  conduct  such  a  study  with  an  existing  mesoscale  meteorological 
observation  network  and  complex  topographical  features. 
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Surface  wind  analysis  data  calculated  by  the  MM5,  AVN,  and  NOGAPS  over  western  Asia  were 
also  compared  with  observed  data,  showing  slightly  better  statistical  results  than  the  BFM  over 
the  Iranian  model  domain.  However,  intercomparisons  of  different  models  operated  over 

different  model  domains  and  periods  were  not  valid.  The  results  should  be  regarded  at  best  as 
qualitative. 

Another  consideration  must  be  taken  into  account  when  comparing  observed  parameters  such  as 
wind  speed  and  vector  components,  temperature,  and  humidity  to  model  calculations  of  these 
parameters.  Observed  data  typically  represent  brief  (~10  min)  averages  of  each  parameter  taken 
from  instruments  which  require  regular  calibration  and  tend  to  characterize  the  time  period  at  the 
top  of  the  hour.  Model  calculations,  on  the  other  hand,  are  more  representative  of  mean  values 
of  the  parameters  over  larger  temporal  and  spatial  scales. 
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List  of  Acronyms 


AD 

Absolute  difference 

AVN 

Aviation  Model 

BFM 

Battlescale  Forecast  Model 

CC 

Correlation  coefficient 

IMETS 

Integrated  Meteorological  System 

MD 

Mean  Difference 

MM5 

Mesoscale  Model  Version  5 

NOGAPS 

Navy  Operational  Global  Atmospheric  Prediction  System 

NORAPS 

Navy  Operational  Regional  Atmospheric  Prediction  System 

RMSE 

Root  mean  square  error 

RMSVE 

Root  mean  square  vector  error 

UTC 

Universal  time  coordinate 
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